Scheduling Parallel-Task Jobs Subject to Packing and Placement Constraints

Jobs in modern parallel-computing frameworks, such as Hadoop and Spark, are subject to several constraints. In these the data typically distributed across a cluster of machines is processed multiple stages. Therefore, tasks that belong same stage (job) have collective completion time determined by slowest task collection. Furthermore, task’s processing machine dependent, each capable at its capacity. “Scheduling Parallel-Task Subject Packing Placement Constraints,” Mehrnoosh Shafiee Javad Ghaderi, approximation algorithms with theoretical guarantees provided solve problem under preemptive nonpreemptive scenarios. The numerical results, using real traffic trace, demonstrate yield significant gains over prior approaches.